About Me

A quick download about me for the curious. I am an Analytics Professional. Meaning I have worn the hats like Chief of Analytics/Sr. Principal Consultant or Sr. Data Architect / Sr. Solutions Architect for signature projects with budgets in dollars of 8, 9,or 10 figures, for entities like at&t, Center Point Energy, some marketing agencies under the Omnicom Umbrella, as well as federal Government with the VA, the Dept. of Health & Human Services, and even a special cabinet committee project for the white house (along with the required security clearances those entail) My educational background is:

• B.S. in Mathematics from University of Texas in Probability, Statistics & Data Analysis & minors in Actuarial Science & Business Foundations.
• MBA from St. Edwards University in Executive & Operations Management.
• Graduated with Honors from the Harvard Business Analytics Program.

I got my start in consulting as an estimator for Drees Custom Homes, the largest private builder in the US, where I was matrix-ed out to the VP of Corporate Operations to modernize organizational workflows through innovation & technology in their 12 divisions nationwide. I helped at&t launch U-verse in one the first mega-budget projects to be managed end to end with analytics at that time. For Center Point Energy, there was a huge smart grid project , funded by the DOE, the State of Texas, the city of Houston & CNP. When it was all said & done my firm had built from the ground up, delivered, & transferred their operational analytics program for electricity distribution & Smart City Management. The technology was combo wireless mesh device network & WiMax system with significant OTA capabilities serving the fourth largest city in the country. No doubt, I was influenced by my father who was a Computer Scientist & my mother who was both an an engineer & Phd. in Neuroscience. Now let’s get started.

Here is a peek preview at a teaser sample chunk of code, using the Iris data set. The outputs are 2 visuals, consider what makes them alike & what differentiates them.

#  Intake ==> ETL ==> [Training Data_idx]

# The Well Known Iris Data Setis provided by R natively
data(iris)
iris$setosa <- iris$Species == "setosa"
iris$virginica <- iris$Species == "virginica"
iris$versicolor <- iris$Species == "versicolor"
iris.train.idx <- sample(x = nrow(iris), size = nrow(iris)*0.5)
iris.train <- iris[iris.train.idx,]
iris.valid <- iris[-iris.train.idx,]

# Pipe: data[]===> Neural_Net_Data-Model_Structure {NN_DM}

library(neuralnet)
iris.net <- neuralnet(setosa + versicolor + virginica ~
  Sepal.Length + Sepal.Width +  Petal.Length + Petal.Width, 
    data = iris.train,
     hidden = c(10,10), 
        err.fct = "ce", 
          linear.output = F, 
            lifesign = "minimal",
              stepmax = 1000000,
                threshold = 0.001,
                   rep = 5, )

## hidden: 10, 10    thresh: 0.001    rep: 1/5    steps:     229    error: 0.00019  time: 0.1 secs
## hidden: 10, 10    thresh: 0.001    rep: 2/5    steps:     425    error: 8e-05    time: 0.23 secs
## hidden: 10, 10    thresh: 0.001    rep: 3/5    steps:     500    error: 6e-05    time: 2.65 secs
## hidden: 10, 10    thresh: 0.001    rep: 4/5    steps:     329    error: 0.00013  time: 0.27 secs
## hidden: 10, 10    thresh: 0.001    rep: 5/5    steps:     426    error: 0.00026  time: 0.25 secs

# Provide a Visual [{NN_DM}]====>{Graph_/Out}

thematic::thematic_on(bg = 'black' , fg = 'white', accent = 'lightgreen' )
NN <- suppressWarnings( NeuralNetTools::plotnet(iris.net ,  
    circle_cex  = 4,
     circle_col = "black",
      bord_col = "lightgreen",
       neg_col = "pink",
        pos_col = "lightgreen",
         cex_val = .75, 
          max_sp = TRUE,
           BIAS = TRUE,
            bias_y = .5,
             pad_x = 1.28,
              alpha = .1,
               rel_rsc = .75,
                alpha_val = .5))

# Predict Code F{NN_DM}(Predict)

iris.prediction <- compute(iris.net, iris.valid[-5:-8])

# Index F{NN_DM}(Predict_idx)
idx <- apply(iris.prediction$net.result, 1, which.max)

predicted <- c('setosa', 'versicolor', 'virginica')[idx]

# Tab Data Structure
NN_DM_Predict__ <- table(predicted, iris.valid$Species)

NN_DM_Predict__ 

##             
## predicted    setosa versicolor virginica
##   setosa         27          0         0
##   versicolor      0         23         0
##   virginica       0          3        22

library(GGally);library(viridis);library(hrbrthemes)

## Loading required package: ggplot2

## Registered S3 method overwritten by 'GGally':
##   method from   
##   +.gg   ggplot2

## Loading required package: viridisLite

# A Visual from the Iris Data Set
suppressWarnings( ggparcoord(
  iris,
   columns = 1:4, 
    groupColumn = 5, 
     order = "anyClass",
      scale = "globalminmax",
        showPoints = TRUE, 
          title = "No scaling",
            alphaLines = 0.3) + 
  
scale_color_viridis(discrete = TRUE) )

Visual: A classifier leverages patterns in data, ponder how the previous 2 visuals might relate to each other.

R Set Up

# options
options(warn = -1)
auto.snapshot <- getOption("renv.config.auto.snapshot")
knitr::opts_chunk$set(message = FALSE)
knitr::opts_chunk$set(warning = FALSE)
knitr::opts_chunk$set(renvfig.showtext = TRUE)

# Initial packages loaded 
librarian::shelf(
#DATA/PARSING/PIPING
library(viridis), library(magrittr), library(dplyr,quietly = T), library(showtext),
# HTML/RMARKDOWN 
library(rmdformats), library(bookdown), library(bslib), library(knitr), library(RColorBrewer),
library(hrbrthemes),
#PARRALELL/GPU/SYSTEM
library(parallelly), library(torch), library(benchmarkme), library(here),
#SPARK/h2o/Sarkling_Water
library(sparklyr), library(sparklyr.nested), library(sparktf), library(torch), library(arrow),
library(sparklyr), library(rsparkling),library(h2o), library(h2otools),
#VISUALS
library(plotly), library(ggplo2), library(ggforce), library(ggpubr), library(GGally),
#TEXT
library(text),
#PYTHON
library(reticulate),
#MODELING 
library(neuralnet), library(usemodels), library(C50), library(dials), library(embed), 
library(xgboost), library(Matrix),
library(tidymodels,quietly = T), library(caret,quietly = T))

PicPath = 'c:/Users/Administrator/Documents/R/NLP/udpipemodels/images_'

thematic::thematic_rmd()
thematic::thematic_on(bg = 'black' , fg = 'white', accent = 'lightgreen' )

# Implementing Timing Features 

knitr::opts_chunk$set(time_it = TRUE)
knitr::knit_hooks$set(time_it = local({
  now <- NULL
  function(before, options) {
    if (before) {
      # record the current time before each chunk
      now <<- Sys.time()
    } 
    else {
      # calculate the time difference after a chunk
      res <- difftime(Sys.time(), now, units = "secs")
      # return a character string to show the time
      paste("Time for this code chunk to run:", base::round(res,2), "seconds")
    }
  }
 }))

# Project
here::dr_here()

# System Architecture 
benchmarkme::get_ram()

## 223 GB

benchmarkme::get_cpu()

## $vendor_id
## [1] "AuthenticAMD"
## 
## $model_name
## [1] "AMD EPYC 7443P 24-Core Processor"
## 
## $no_of_cores
## [1] 48

# FROM TORCH
library(torch)

cuda_device_count()

## [1] 1

 cuda_current_device()

## [1] 0

   cuda_is_available()

## [1] TRUE

     cuda_runtime_version()

## [1] '11.8.0'

cuda_get_device_capability(device = cuda_current_device())

## Major Minor 
##     7     5

II . A Neural Net in Action

library(neuralnet); library(caret)

# Iris has 3 species & 4 attributes to model 
data("iris"); str(iris); set.seed(19)

## 'data.frame':    150 obs. of  5 variables:
##  $ Sepal.Length: num  5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ...
##  $ Sepal.Width : num  3.5 3 3.2 3.1 3.6 3.9 3.4 3.4 2.9 3.1 ...
##  $ Petal.Length: num  1.4 1.4 1.3 1.5 1.4 1.7 1.4 1.5 1.4 1.5 ...
##  $ Petal.Width : num  0.2 0.2 0.2 0.2 0.2 0.4 0.3 0.2 0.2 0.1 ...
##  $ Species     : Factor w/ 3 levels "setosa","versicolor",..: 1 1 1 1 1 1 1 1 1 1 ...

#Intake ==> ETL ==> [Training Data_idx]
 
indexes = createDataPartition(iris$Species, p = .85, list = F)

train = iris[indexes, ] ; test = iris[-indexes, ] 

xtest = test[, -5] ; ytest = test[, 5]



# Pipe: data[]===> Neural_Net_Data-Model_Structure {NNet_DM}

nnet =  neuralnet(
                  Species~., 
                  train,
                  hidden = c(4,3),
                  linear.output = FALSE
                 )


# Provide a Visual [{NNet_DM}]====>{Graph_/Out}

plot(
     nnet,
     col.entry.synapse = "grey1", 
     col.entry = "brown",
     col.hidden = "lightgreen", 
     col.hidden.synapse = "grey",
     col.out = "red", 
     col.out.synapse = "black",
     col.intercept = "black",
     information = TRUE,
     fontsize = 9,
     arrow.length = .15,
     rep = "best"
     )

Time for this code chunk to run: 2.19 seconds

thematic::thematic_on(
                      bg = 'black' , 
                      fg = 'white', 
                      accent = 'lightgreen'
                     )

NN <- suppressWarnings( 
                        NeuralNetTools::plotnet(nnet, 
                                      circle_cex    = 4,
                                      circle_col = "black",
                                      bord_col = "lightgreen",
                                      neg_col = "pink",
                                      pos_col = "lightgreen",
                                      cex_val = .55, 
                                      alpha_val = .5)
                      )

Time for this code chunk to run: 0.74 seconds

The learning rate in machine learning is generally referred to by the hyper-parameter that determines the step size at each iteration while moving toward a minimum of a loss function. It controls how much the model’s weights are updated during training. An analytical method cannot be used to calculate the weights of a neural network. Instead, the weights must be discovered using stochastic gradient descent, an empirical optimization approach. In simpler terms, the stochastic gradient descent algorithm is used to train deep learning rate neural networks. A typical value for the learning rate is between 0.0 and 1.0. In actuality though, we have 2 kinds of learning or updating of our models.

• Machine learnable parameters – The parameters that the algorithms learn/estimate on their own during training for a particular data set.

• Hyper-parameters are variables that machine learning engineers or data scientists provide precise values to regulate how algorithms learn and modify the model’s performance.

An analytical method cannot be used to calculate the weights of a neural network. Instead, the weights must be discovered using stochastic gradient descent, an empirical optimization approach. In simpler terms, the stochastic gradient descent algorithm is used to train deep learning rate neural networks So what does Learning Rate look like?

library(ggplot2)
library(ggpubr)
library(plotly)

# Neural networks are neurons that are sequentially connected. Each of this sequence is called neuron layer.

set.seed(19)

circular <- function(x, R, centerX=0, centerY=0){
r = R * sqrt(runif(x))
theta = runif(x) * 2 * pi
x = centerX + r * cos(theta)
y = centerY + r * sin(theta)

z = data.frame(x = x, y = y)
return(z)
}

data1 <- circular(150,0.5)
data2 <- circular(150,1.5)

data1$Y <- 1
data2$Y <- 0
data_c <- rbind(data1,data2)


color <- c("yellow", "lightgreen")
plot(data_c, col = factor(data_c$Y))

rm(data1,data2, circular)

# Here is what we have so far:
library(ggplot2)
Plot1 <- ggplot(data_c ,aes(x,y, col = as.factor(Y))) + geom_point()

#Classification problem to solve with a neural network built from scratch in R
X <- as.matrix(data_c[,1:2])
Y <- as.matrix(data_c[,3])


# Its now time to create the structure of the neural network. To do so, we will implement a Class-Neuron which we can iterate through.
# This class will have the following parameters: Number of neurons on the layers 1,  Number of neurons on layer 2, the number of connections

neuron <- setRefClass(
                      "neuron",
                      fields = list(
                      fun_act = "list",
                      number_connections = "numeric",
                      number_neurons = "numeric",
                      W = "matrix",
                      b = "numeric"
                     ),
                     
 # Here is our activation function
methods = list(
initialize = function(
                      fun_act, 
                      number_connections,
                      number_neurons)
    {
      fun_act <<- fun_act
      number_connections <<- number_connections
      number_neurons <<- number_neurons
      
      W <<- matrix(runif(number_connections*number_neurons),nrow = number_connections)
     
       b <<- runif(number_neurons)
    }
  )
)


# Retrieve the activation functions & their derivatives ==> very useful when implementing gradient descent. 

# Sigmoid activation function and it’s derivative as follows:
sigmoid = function(x) 
  {
  y = list() 
  y[[1]] <- 1 / (1 + exp(-x))
  y[[2]] <- x * (1 - x)
  return(y)
  }

x <- seq(-5, 5, 0.01)
#s <- plot(x, sigmoid(x)[[1]], col = 'lightgreen')

# Relu activation function and it’s derivative as follows:
relu <- function(x)
  {
  y <- list()
  y[[1]] <- ifelse(x < 0,0,x)
  y[[2]] <- ifelse(x < 0,0,1)
  return(y)
  }

#r = plot(x, relu(x)[[1]], col = 'lightgreen')


# The weights for the incoming connections (W):, Bias(b): is added to every neuron after being weighted with W. 

n = ncol(X)                                     #Number of neurons Primary level 
capas = c(n, 4, 8, 1)                           # Number of neurons hidden level 
Function = list(sigmoid, relu, sigmoid)                   # Activation Functions
fuscia <- list()


for (i in 1:(length(capas) - 1))
  {
    fuscia[[i]] <- neuron$new(Function[i],capas[i], capas[i + 1])
  }


library(crayon)

cli::cli_alert_info("Class Definition: {fuscia}")
cat(green("\n  Class Instance created"));fuscia

## 
##   Class Instance created

## [[1]]
## Reference class object of class "neuron"
## Field "fun_act":
## [[1]]
## function(x) 
##   {
##   y = list() 
##   y[[1]] <- 1 / (1 + exp(-x))
##   y[[2]] <- x * (1 - x)
##   return(y)
##   }
## 
## Field "number_connections":
## [1] 2
## Field "number_neurons":
## [1] 4
## Field "W":
##            [,1]      [,2]      [,3]       [,4]
## [1,] 0.07396211 0.5202001 0.8658044 0.90293385
## [2,] 0.51024832 0.5226468 0.8529940 0.03439049
## Field "b":
## [1] 0.4044959 0.9406215 0.5616665 0.1888222
## 
## [[2]]
## Reference class object of class "neuron"
## Field "fun_act":
## [[1]]
## function(x)
##   {
##   y <- list()
##   y[[1]] <- ifelse(x < 0,0,x)
##   y[[2]] <- ifelse(x < 0,0,1)
##   return(y)
##   }
## 
## Field "number_connections":
## [1] 4
## Field "number_neurons":
## [1] 8
## Field "W":
##           [,1]      [,2]      [,3]      [,4]      [,5]      [,6]      [,7]
## [1,] 0.2812473 0.6275187 0.4906343 0.9806989 0.9252840 0.7751652 0.2626343
## [2,] 0.3528488 0.5526032 0.4955991 0.9231114 0.5929445 0.8522817 0.7870782
## [3,] 0.1239218 0.4302792 0.5403289 0.7110307 0.3400002 0.3930699 0.9133510
## [4,] 0.6935446 0.3268714 0.7616799 0.7535075 0.2026201 0.2313034 0.1823203
##            [,8]
## [1,] 0.02728611
## [2,] 0.03334917
## [3,] 0.21608093
## [4,] 0.45169708
## Field "b":
## [1] 0.650707836 0.009437195 0.341280788 0.611787775 0.629912437 0.076977954
## [7] 0.020569250 0.242895248
## 
## [[3]]
## Reference class object of class "neuron"
## Field "fun_act":
## [[1]]
## function(x) 
##   {
##   y = list() 
##   y[[1]] <- 1 / (1 + exp(-x))
##   y[[2]] <- x * (1 - x)
##   return(y)
##   }
## 
## Field "number_connections":
## [1] 8
## Field "number_neurons":
## [1] 1
## Field "W":
##           [,1]
## [1,] 0.9752737
## [2,] 0.5134806
## [3,] 0.9814409
## [4,] 0.3123976
## [5,] 0.6806086
## [6,] 0.9050108
## [7,] 0.6584775
## [8,] 0.9161916
## Field "b":
## [1] 0.569508

fuscia

## [[1]]
## Reference class object of class "neuron"
## Field "fun_act":
## [[1]]
## function(x) 
##   {
##   y = list() 
##   y[[1]] <- 1 / (1 + exp(-x))
##   y[[2]] <- x * (1 - x)
##   return(y)
##   }
## 
## Field "number_connections":
## [1] 2
## Field "number_neurons":
## [1] 4
## Field "W":
##            [,1]      [,2]      [,3]       [,4]
## [1,] 0.07396211 0.5202001 0.8658044 0.90293385
## [2,] 0.51024832 0.5226468 0.8529940 0.03439049
## Field "b":
## [1] 0.4044959 0.9406215 0.5616665 0.1888222
## 
## [[2]]
## Reference class object of class "neuron"
## Field "fun_act":
## [[1]]
## function(x)
##   {
##   y <- list()
##   y[[1]] <- ifelse(x < 0,0,x)
##   y[[2]] <- ifelse(x < 0,0,1)
##   return(y)
##   }
## 
## Field "number_connections":
## [1] 4
## Field "number_neurons":
## [1] 8
## Field "W":
##           [,1]      [,2]      [,3]      [,4]      [,5]      [,6]      [,7]
## [1,] 0.2812473 0.6275187 0.4906343 0.9806989 0.9252840 0.7751652 0.2626343
## [2,] 0.3528488 0.5526032 0.4955991 0.9231114 0.5929445 0.8522817 0.7870782
## [3,] 0.1239218 0.4302792 0.5403289 0.7110307 0.3400002 0.3930699 0.9133510
## [4,] 0.6935446 0.3268714 0.7616799 0.7535075 0.2026201 0.2313034 0.1823203
##            [,8]
## [1,] 0.02728611
## [2,] 0.03334917
## [3,] 0.21608093
## [4,] 0.45169708
## Field "b":
## [1] 0.650707836 0.009437195 0.341280788 0.611787775 0.629912437 0.076977954
## [7] 0.020569250 0.242895248
## 
## [[3]]
## Reference class object of class "neuron"
## Field "fun_act":
## [[1]]
## function(x) 
##   {
##   y = list() 
##   y[[1]] <- 1 / (1 + exp(-x))
##   y[[2]] <- x * (1 - x)
##   return(y)
##   }
## 
## Field "number_connections":
## [1] 8
## Field "number_neurons":
## [1] 1
## Field "W":
##           [,1]
## [1,] 0.9752737
## [2,] 0.5134806
## [3,] 0.9814409
## [4,] 0.3123976
## [5,] 0.6806086
## [6,] 0.9050108
## [7,] 0.6584775
## [8,] 0.9161916
## Field "b":
## [1] 0.569508

cat(yellow("\n This is a field in class neuron\n",neuron$fields(), "\n"))

## 
##  This is a field in class neuron
##  list 
##  
##  This is a field in class neuron
##  numeric 
##  
##  This is a field in class neuron
##  numeric 
##  
##  This is a field in class neuron
##  matrix 
##  
##  This is a field in class neuron
##  numeric

# create a new plotting window and set the plotting area into a 1*2 array
par(mfrow = c(1, 2))
suppressWarnings(plot(x, relu(x)[[1]], col = 'lightgreen', title = 'relu'))
suppressWarnings(plot(x, sigmoid(x)[[1]], col = 'lightgreen', title = 'rsigmoid'))

suppressWarnings(ggplotly(Plot1))

cli::cli_alert_success(cat(yellow("\n Thus, we have a S4 class fuscia neuron, next we initialize the parameters W and b with the function runif   ")))

## 
##  Thus, we have a S4 class fuscia neuron, next we initialize the parameters W and b with the function runif

Time for this code chunk to run: 1.95 seconds

Output: Be sure to see the class definition with data & its fields.

Activation Function

Activation / Loss Functions

Loss Function: A function that returns the cost associated with the model and measures how well our model is doing on the training data. If the cost is too high, it means that the predictions by our model are deviating too much from the observed data. In any machine learning algorithm, our ultimate mission is to minimize the loss function, sometimes referred to as the evolution of the error.

# Training the Neural Network: Forward Propagation
# When doing the forward pass we will store the results of the neuron before and after applying the activation functions (z).
#  This will be useful on back propagation.

set.seed(19)
conflicted::conflicts_prefer(base::`%*%`)

entry <- function(fuscia, X,Y, cost){
out = list()
out[[1]] <- append(list(matrix(0,ncol = 2,nrow = 1)), list(X))
 i = 0
  for (i in c(1:(length(fuscia)))) {
    z = list((out[[length(out)]][[2]] %*% fuscia[[i]]$W + fuscia[[i]]$b))
    a = list(fuscia[[i]]$fun_act[[1]](z[[1]])[[1]])
    out[[i + 1]] <- append(z,a)
  }
return(out)
}

cost <- function(Yp,Yr){
y <- list()
y[[1]] <- mean((Yp - Yr)^2)
y[[2]] <- (Yp - Yr)
return(y)

forward <- entry(fuscia, X,Y, cost)
head(forward[[4]][[2]]) 
}

#- Implemented a neural network ==> Examine Performance
#- Implementing front and back propagation w/ gradient descent on the same function

fuscia_neuronal <- function(red, X,Y, cost,lr = 0.05){
## Front Prop
out = list()
out[[1]] <- append(list(matrix(0,ncol = 4,nrow = 1)), list(X))
 i = 0
  for (i in c(1:(length( fuscia)))) 
    {
    z = list((out[[length(out)]][[2]] %*% fuscia[[i]]$W + fuscia[[i]]$b))
    a = list( fuscia[[i]]$fun_act[[1]](z[[1]])[[1]])
    out[[i + 1]] <- append(z,a)
    }

## Back-prop & Gradient Descent
delta <- list() 

for (i in rev(1:length( fuscia))) 
  {
  z = out[[i + 1]][[1]]
  a = out[[i + 1]][[2]]
  if (i == length( fuscia)) 
    {
    delta[[1]] <- cost(a,Y)[[2]] * fuscia[[i]]$fun_act[[1]](a)[[2]]
    } 
  else{
    delta <- list(delta[[1]] %*% W_temp * fuscia[[i]]$fun_act[[1]](a)[[2]],delta)
      }
  W_temp = t(fuscia[[i]]$W)
  fuscia[[i]]$b <-  fuscia[[i]]$b - mean(delta[[1]]) * lr
  fuscia[[i]]$W <-  fuscia[[i]]$W - t(out[[i]][[2]]) %*% delta[[1]] * lr
 }
return(out[[length(out)]][[2]])
}
  
 # Test neural network n = 2500
result <- fuscia_neuronal(fuscia, X,Y, cost)
dim(result)

## [1] 300   1

i_ = 0

for (i_ in seq(2500)) {
  Yt = fuscia_neuronal(fuscia, X,Y, cost, lr = 0.01)
  if (i_ %% 25 == 0) 
    {
    if (i_ == 25) 
      {
      iteration <- i_
      error <- cost(Yt,Y)[[1]]
      }
    else
      {
       iteration <- c(iteration,i_)
       error <- c(error,cost(Yt,Y)[[1]])      
      }
    }
}


# visualize how the error of our neural network has evolved:

thematic::thematic_on(bg = 'black' , fg = 'lightgreen', accent = 'pink' )
plot(error)

library(plotly)

thematic::thematic_on(bg = 'black' , fg = 'white', accent = 'lightgreen' )

Loss_model <-  plot_ly(x = iteration, 
                       y =  error, 
                       type = 'scatter', 
                       mode = 'lines',
                       line = list(color = 'lightgreen'))

  Loss_model |> 
    plotly::layout(paper_bgcolor = "black",
      title = "Error for the  Fuscia Nueronal",
       font = list(color = 'white'),
        plot_bgcolor = "grey2",
         xaxis = list(title = "Iterations",
                      gridcolor = 'grey',
                      showgrid = TRUE,
                      showline = TRUE,
                      showticklabels = TRUE,
                      tickcolor = 'white',
                      ticks = 'outside',
                      zeroline = FALSE),
         yaxis = list(title = "Loss",
                      gridcolor = 'grey',
                      showgrid = TRUE,
                      showline = TRUE,
                      showticklabels = TRUE,
                      tickcolor = 'white',
                      ticks = 'outside',
                      zeroline = FALSE))

Time for this code chunk to run: 4.4 seconds

III. The Effect of Gradient Descent

Learning Rate: This is the hyperparameter that determines the steps the gradient descent algorithm takes. Gradient Descent is too sensitive to the learning rate. If it is too big, the algorithm may bypass the local minimum and overshoot. If it too small, it might increase the total computation time to a very large extent. We will see the effect of the learning rate in depth later in the article.

Gradient: Basically, it is a measure of the steepness of a slope. And technically, when we sum up all the first-order derivatives of all the variables in a function, it gives us gradient. For example, if we consider linear regression, we have two parameters, slope, and the intercept, to minimize. So, we calculate derivatives w.r.t. both slope & the intercept and then sum them up to get the gradient for it.

Descent: To optimize parameters, we need to minimize errors. The aim of the gradient descent algorithm is to reach the local minimum (though we always aim to reach the global minimum of the function. But if a gradient descent algorithm once attains the local minimum, it is nearly impossible to reach the global minimum.). The algorithm accomplishes this by an iterative process of calculating step size at every iteration. And, this iterative calculation of step size to reach a local minimum (or in other words, descending to the point of minimum) is known as the descent (Enough of that going down the hill example).

# Use Oranges Data Set
library(RColorBrewer)
data("Orange")

# Determine number of iterations
niter <- 85

# Determine learning rate / step size
alpha <- 1e-4

set.seed(19)
b0 <- rnorm(1) # intercept
b1 <- rnorm(1) # slope

# Set palette
cols <- colorRampPalette(rev(brewer.pal(n = 7, name = "RdBu")))(niter)
cols2 <- colorRampPalette((brewer.pal(n = 7, name = "RdBu")))(niter)

# Plot
 plotly::layout(matrix(c(1,1,2,2,3), nrow = 1, ncol = 5))
plot(age ~ circumference, data = Orange, 
     pch = 16,
     xlab = "Circumference (mm)",
     ylab = "Age (days)",
     col = cols2,
     col.axis = 'gold')


# Perform gradient descent
slopes <- rep(NA, niter)
intercepts <- rep(NA, niter)

for (i in 1:niter) {  
# prediction
  y <- b0 + b1 * Orange$circumference
  
# b0 = b0 - dJ/da * alpha
  b0 <- b0 - sum(y - Orange$age) / nrow(Orange) * alpha
  
# b1 = b1 - dJ/db * alpha
  b1 <- b1 - sum((y - Orange$age) * Orange$circumference) / nrow(Orange) * alpha
  abline(a = b0, b = b1, col = cols[i], lty = 2)
  
# Save estimates over all iterations
  intercepts[i] <- b0
  slopes[i] <- b1
}

title("Regression Fit")


# Cost function contour
allCombs <- expand.grid(
                        b0 = seq(-50, 50, 
                        length.out = 100),
                        b1 = seq(7.5, 8.5, 
                        length.out = 100)
                        )

res <- matrix(NA, 100, 100)
# a by rows, b by cols
for (i in 1:nrow(allCombs))
   {
    y <- allCombs$b0[i] + allCombs$b1[i] * Orange$circumference
    res[i] <- sum((y - Orange$age)**2)/(2*nrow(Orange))
   }

par( bg = "black", fg = "white")



# Plot MSE contour Diagram
contour(
        t(res), 
        xlab = expression(beta[1]),
        ylab = expression(beta[0]), 
        axes = F,
        nlevels = 25,
        col = cols2,
        col.lab = 'beige'
       )
  
axis(1, at = c(0, .5, 1), labels = c(7.5, 8, 8.5), col = 'lightyellow' , col.axis = 'yellow')
axis(2, at = c(0, .5, 1), labels = c(-50, 0, 50) , col = 'lightyellow' , col.axis = 'yellow')

points(
       (slopes - min(allCombs$b1)) / diff(range(allCombs$b1)),
       (intercepts - min(allCombs$b0)) / diff(range(allCombs$b0)),
        pch = 19, 
        cex = 1.5,
        col = cols,
        col.lab = 'beige'
  
)

title("MSE contour")

# Add colorbar
z = matrix(1:niter, nrow = 1)
image(
      1, 
      1:niter, z,
      col = cols, 
      axes = FALSE,
      xlab = "", 
      ylab = ""
      )

title("Iteration No.", col = 'white')
axis(2, at = c(1, seq(5, niter, by = 5)))

Time for this code chunk to run: 0.55 seconds

IV. Exploratory Data Analysis (EDA) & ERM

library(plotly)
library(ggplot2)

# LM Plot 
p <- ggplot(Orange, aes(circumference, age))

# Use jitter to reduce overplotting
p <- p + geom_jitter(position = position_jitter(width = 0.5, height = 0.5)) + 
  geom_density_2d(aes(color = ..level..)) + 
  scale_color_viridis_c() + 
  geom_smooth(method = 'lm', formula =  y~x)



# Residuald Plot
library(broom)

mod <- lm(Orange$age ~ Orange$circumference )
df <- augment(mod)
g <- ggplot(df, aes(x = .fitted, y = .resid)) 

# Use jitter to reduce overplotting
g <- g + geom_jitter(position = position_jitter(width = 0.5, height = 0.5)) + 
     geom_density_2d(aes(color = ..level..)) + 
     scale_color_viridis_c() + 
     geom_smooth(method = 'lm', formula =  y~x)

par(c(1,2))

## NULL

ggplotly(p)

ggplotly(g)

Time for this code chunk to run: 2.43 seconds

Principal Component Analysis Interactive Plot

The EDA could include the Principal component analysis (PCA), which is a projection of the 4-dimensional iris flowering data on 2-dimensional space using the first two principal components. We can see that the first principal component alone is useful in distinguishing the three species.

• We could use simple rules like this: If PC1 < -1, then Iris Setosa.
• If PC1 > 1.5 then Iris Virginica.
• If -1 < PC1 < 1, then Iris Versicolor.

It is these patterns & others which we are leveraging in NN’s.

library(plotly)

data(iris)

axis = list(showline = FALSE,
            zeroline = FALSE,
            gridcolor = '#ffff',
            ticklen = 4,
            titlefont = list(size = 13))


fig  <- iris %>%
  plot_ly()

fig <- fig %>%
  add_trace(
            type = 'splom',
            dimensions = list(
           list(label = 'sepal length', values = ~Sepal.Length),
           list(label = 'sepal width', values = ~Sepal.Width),
           list(label = 'petal length', values = ~Petal.Length),
           list(label = 'petal width', values = ~Petal.Width)
           ),
 
           color = ~ Species, 
           colors = c('#636EFA','#EF553B','#00CC96') ,
           marker = list(size = 7,
                         line = list(
                                     width = 1,
                                     color = 'rgb(230,230,230)'
                                    )
                        )
          )    

fig <-  fig %>% plotly::style(diagonal = list(visible = FALSE))

fig <- fig %>%
plotly::layout(
                hovermode = 'closest',
                dragmode = 'select',
                plot_bgcolor = "lightgrey",
                xaxis = list(
                             domain = NULL, 
                             showline = F, 
                             zeroline = F,
                             gridcolor = '#ffff', 
                             ticklen = 4
                            ),
                yaxis = list(
                             domain = NULL, 
                             showline = F,
                             zeroline = F, 
                             gridcolor = '#ffff', 
                             ticklen = 4
                            ),
                xaxis2 = axis,
                xaxis3 = axis,
                xaxis4 = axis,
                yaxis2 = axis,
                yaxis3 = axis,
                yaxis4 = axis 
             )

# Create scatterplots all pairwise combinations for the 4 variables 

fig

pca <- prcomp(iris[, 1:4], scale = TRUE) 
pca 

## Standard deviations (1, .., p=4):
## [1] 1.7083611 0.9560494 0.3830886 0.1439265
## 
## Rotation (n x k) = (4 x 4):
##                     PC1         PC2        PC3        PC4
## Sepal.Length  0.5210659 -0.37741762  0.7195664  0.2612863
## Sepal.Width  -0.2693474 -0.92329566 -0.2443818 -0.1235096
## Petal.Length  0.5804131 -0.02449161 -0.1421264 -0.8014492
## Petal.Width   0.5648565 -0.06694199 -0.6342727  0.5235971

# Have a look at the results.  
# extract first two columns and convert to data frame 

pcaData <- as.data.frame(pca$x[, 1:2]) 
pcaData <- cbind(pcaData, iris$Species) 
colnames(pcaData) <- c("PC1", "PC2", "Species")   

# compute % variances 

percentVar <- base::round(100 * summary(pca)$importance[2, 1:2], 0)  


# starting ggplot2  add data points, x label, y label  

library(ggplot2)

q <-  ggplot(pcaData, 
             aes(PC1, 
                 PC2, 
                 color = Species, 
                 shape = Species)) +  
                 geom_point(size = 2) + 
                 xlab(paste0("PC1: ", percentVar[1], "% variance")) +
                 ylab(paste0("PC2: ", percentVar[2], "% variance")) +                  ggtitle("Principal component analysis (PCA)") 

ggplotly(q)

Time for this code chunk to run: 0.55 seconds

We can gain many insights from EDA. Interactive analytics allows for a better experience for the customer & as well as the provider. It promotes engagement, & creates data opportunities that a static experience may miss on. For example take the visual below.

• The 150 flowers in the rows are organized into different clusters.

• Setosa samples obviously formed a unique cluster, characterized by smaller (blue) petal length, petal width, and sepal length.

• The other two subspecies are not clearly separated but we can notice that some I.

• Virginica samples form a small sub cluster showing bigger petals.

• The columns are also organized into dendrograms, which clearly suggest that petal length and petal width are highly correlated. Thus we can communicate partly how these attributes are considered through our NN model weights, & what factors contribute to their magnitudes (strength) & charge,( i.e positive / negative ).

• Heat Map - The following code will build the tree divided heat map with the Iris data set.

  library(pheatmap) ma <- as.matrix(iris[, 1:4])
  
  # assign row names in the matrix convert to matrix
  row.names(ma) <- row.names(iris)
   par( bg = “black”, fg = “white”)
  
  pheatmap_ <- pheatmap(ma, scale = “column”,
  clustering_method = “average”,
  annotation_row = iris[, 5, drop = FALSE],
  show_rownames = FALSE,
  width = 7,
  height = 5,)

_____________________________________________________________________________________________________

V. Sparkling Water / h2o ai

Above; H2O Flow is an open-source user interface for H2O. It is a web-based interactive environment that allows you to combine code execution, text, mathematics, plots, and rich media in a single document.

• Install sparkling water : install.packages(“C:/Users/Administrator/AppData/Local/spark/spark-3.5.0-bin-hadoop3/sparkling-water-3.46.0.3-1-3.5/rsparkling_3.46.0.3-1-3.5.tar.gz”)

• Install sparkling water - in project : renv::install(‘C:/Users/Administrator/AppData/Local/spark/spark-3.5.0-bin-hadoop3/sparkling-water-3.46.0.3-1-3.5/rsparkling_3.46.0.3-1-3.5.tar.gz’)

  Initialization

library(sparklyr);library(rsparkling);library(h2o);
library(h2otools);library(dplyr)

# SPARK_RAPIDS_PLUGIN_JAR = 'C:/Users/Administrator/AppData/Local/spark/spark-3.5.0-bin-hadoop3/GPU/rapids-4-spark_2.12-24.06.0.jar'
# 
# SPARK_SQL_PLUGIN_JAR = 'C:/Users/Administrator/AppData/Local/spark/spark-3.5.0-bin-hadoop3/GPU/cudf-24.06.0.jar'

#Pipe:data[config]==>{S_data[config.default]} Initialize w/ defaults 

config <- spark_config() 
options(sparklyr.log.console = TRUE)

#Pipe:data[config]==>{S_data[config.user]} Initialize w/ user.settings

config["sparklyr.shell.driver-memory"] <- "32g"        
config["sparklyr.connect.cores.local"] <- 32        
config["sparklyr.connect.timeout"] <- 200            
config["sparklyr.log.console"] <- TRUE
config["spark.driver.extraJavaOptions"] <- "-Duser.timezone = UTC"
config["spark.executor.memory"] <- '32G' 
config["spark.executor.memoryOverhead"] <- '2G'
config["spark.executor.resource.gpu.amount"] <- 1
config["spark.sql.session.timeZone"] <- 'UTC'
config["spark.sql.files.maxPartitionBytes"] <- '512m'

#config["spark.executor.resource.gpu.amount"] <- 1
#config["spark.task.resource.gpu.amount"] <- .0125
#config["spark.task.cpus"] <- 1
#config["spark.rapids.sql.enabled"] <- TRUE
#config["spark.rapids.memory.gpu.maxAllocFraction"] <- .8
#config["spark.jars"] = 'SPARK_SQL_PLUGIN_JAR,SPARK_RAPIDS_PLUGIN_JAR'
#config["spark.rapids.sql.concurrentGpuTasks"] <- 2
#config["spark.executor.resource.gpu.discoveryScript"] <- GET_GPU
#config["spark.sql.shuffle.partitions.local"] <- 8
#config["spark.plugins"] <- 'com.nvidia.spark.SQLPlugin'
#coonnect to local cluster with custom configuration

sc <- sparklyr::spark_connect(master = "local", version = "3.5.0", config = config )

#Pipe:data[config]==>{S_data[config]}===>{S_data[config].h2o}
h2oConf <- H2OConf()
config["h2oConf.setNthreads"] <- 40
hc <- H2OContext.getOrCreate(h2oConf)

##  Connection successful!
## 
## R is connected to the H2O cluster: 
##     H2O cluster uptime:         11 seconds 995 milliseconds 
##     H2O cluster timezone:       America/Chicago 
##     H2O data parsing timezone:  UTC 
##     H2O cluster version:        3.46.0.3 
##     H2O cluster version age:    1 month 
##     H2O cluster name:           sparkling-water-Administrator_local-1720774146110 
##     H2O cluster total nodes:    1 
##     H2O cluster total memory:   32.00 GB 
##     H2O cluster total cores:    48 
##     H2O cluster allowed cores:  48 
##     H2O cluster healthy:        TRUE 
##     H2O Connection ip:          127.0.0.1 
##     H2O Connection port:        54324 
##     H2O Connection proxy:       NA 
##     H2O Internal Security:      FALSE 
##     R Version:                  R version 4.4.0 (2024-04-24 ucrt) 
## 
## Reference class object of class "H2OContext"
## Field "jhc":
## <jobj[22]>
##   ai.h2o.sparkling.H2OContext
##   
## Sparkling Water Context:
##  * Sparkling Water Version: 3.46.0.3-1-3.5
##  * H2O name: sparkling-water-Administrator_local-1720774146110
##  * cluster size: 1
##  * list of used nodes:
##   (executorId, host, port)
##   ------------------------
##   (0,192.168.1.19,54321)
##   ------------------------
## 
##   Open H2O Flow in browser: http://127.0.0.1:54324 (CMD + click in Mac OSX)
## 
## 

h2o.networkTest()

## Network Test: Launched from C-Cluster/192.168.1.19:54321
##                         destination                 1_bytes          1024_bytes
## 1     all - collective bcast/reduce   2.512 msec,  796  B/S  97 usec, 20.1 MB/S
## 2 self C-Cluster/192.168.1.19:54321    1.128 msec, 1.7 KB/S  504 usec, 3.9 MB/S
##          1048576_bytes
## 1  82 usec, 23.57 GB/S
## 2  598 usec, 3.26 GB/S

Time for this code chunk to run: 25.4 seconds

Spark UI w/ Sparkling Water tab upon successful connection

Sparkling Water allows users to combine the fast, scalable machine learning algorithms of H2O with the capabilities of Spark. With Sparkling Water, users can drive computation from Scala/R/Python and utilize the H2O Flow UI, providing an ideal machine learning platform for application developers.

Spark is an elegant and powerful general-purpose, open-source, in-memory platform with tremendous momentum. H2O is an in-memory application for machine learning that is reshaping how people apply math and predictive analytics to their business problems.

So lets dance now, we’ll build models, predict, & evaluate different classes of models using ML.

# Intake ==> ETL ==> [Connections Sockets]

h2o.networkTest()

## Network Test: Launched from C-Cluster/192.168.1.19:54321
##                         destination                 1_bytes          1024_bytes
## 1     all - collective bcast/reduce     157 usec, 12.4 KB/S  854 usec, 2.3 MB/S
## 2 self C-Cluster/192.168.1.19:54321   4.198 msec,  476  B/S 189 usec, 10.3 MB/S
##              1048576_bytes
## 1   2.678 msec, 746.6 MB/S
## 2      649 usec, 3.01 GB/S

iris

##     Sepal.Length Sepal.Width Petal.Length Petal.Width    Species
## 1            5.1         3.5          1.4         0.2     setosa
## 2            4.9         3.0          1.4         0.2     setosa
## 3            4.7         3.2          1.3         0.2     setosa
## 4            4.6         3.1          1.5         0.2     setosa
## 5            5.0         3.6          1.4         0.2     setosa
## 6            5.4         3.9          1.7         0.4     setosa
## 7            4.6         3.4          1.4         0.3     setosa
## 8            5.0         3.4          1.5         0.2     setosa
## 9            4.4         2.9          1.4         0.2     setosa
## 10           4.9         3.1          1.5         0.1     setosa
## 11           5.4         3.7          1.5         0.2     setosa
## 12           4.8         3.4          1.6         0.2     setosa
## 13           4.8         3.0          1.4         0.1     setosa
## 14           4.3         3.0          1.1         0.1     setosa
## 15           5.8         4.0          1.2         0.2     setosa
## 16           5.7         4.4          1.5         0.4     setosa
## 17           5.4         3.9          1.3         0.4     setosa
## 18           5.1         3.5          1.4         0.3     setosa
## 19           5.7         3.8          1.7         0.3     setosa
## 20           5.1         3.8          1.5         0.3     setosa
## 21           5.4         3.4          1.7         0.2     setosa
## 22           5.1         3.7          1.5         0.4     setosa
## 23           4.6         3.6          1.0         0.2     setosa
## 24           5.1         3.3          1.7         0.5     setosa
## 25           4.8         3.4          1.9         0.2     setosa
## 26           5.0         3.0          1.6         0.2     setosa
## 27           5.0         3.4          1.6         0.4     setosa
## 28           5.2         3.5          1.5         0.2     setosa
## 29           5.2         3.4          1.4         0.2     setosa
## 30           4.7         3.2          1.6         0.2     setosa
## 31           4.8         3.1          1.6         0.2     setosa
## 32           5.4         3.4          1.5         0.4     setosa
## 33           5.2         4.1          1.5         0.1     setosa
## 34           5.5         4.2          1.4         0.2     setosa
## 35           4.9         3.1          1.5         0.2     setosa
## 36           5.0         3.2          1.2         0.2     setosa
## 37           5.5         3.5          1.3         0.2     setosa
## 38           4.9         3.6          1.4         0.1     setosa
## 39           4.4         3.0          1.3         0.2     setosa
## 40           5.1         3.4          1.5         0.2     setosa
## 41           5.0         3.5          1.3         0.3     setosa
## 42           4.5         2.3          1.3         0.3     setosa
## 43           4.4         3.2          1.3         0.2     setosa
## 44           5.0         3.5          1.6         0.6     setosa
## 45           5.1         3.8          1.9         0.4     setosa
## 46           4.8         3.0          1.4         0.3     setosa
## 47           5.1         3.8          1.6         0.2     setosa
## 48           4.6         3.2          1.4         0.2     setosa
## 49           5.3         3.7          1.5         0.2     setosa
## 50           5.0         3.3          1.4         0.2     setosa
## 51           7.0         3.2          4.7         1.4 versicolor
## 52           6.4         3.2          4.5         1.5 versicolor
## 53           6.9         3.1          4.9         1.5 versicolor
## 54           5.5         2.3          4.0         1.3 versicolor
## 55           6.5         2.8          4.6         1.5 versicolor
## 56           5.7         2.8          4.5         1.3 versicolor
## 57           6.3         3.3          4.7         1.6 versicolor
## 58           4.9         2.4          3.3         1.0 versicolor
## 59           6.6         2.9          4.6         1.3 versicolor
## 60           5.2         2.7          3.9         1.4 versicolor
## 61           5.0         2.0          3.5         1.0 versicolor
## 62           5.9         3.0          4.2         1.5 versicolor
## 63           6.0         2.2          4.0         1.0 versicolor
## 64           6.1         2.9          4.7         1.4 versicolor
## 65           5.6         2.9          3.6         1.3 versicolor
## 66           6.7         3.1          4.4         1.4 versicolor
## 67           5.6         3.0          4.5         1.5 versicolor
## 68           5.8         2.7          4.1         1.0 versicolor
## 69           6.2         2.2          4.5         1.5 versicolor
## 70           5.6         2.5          3.9         1.1 versicolor
## 71           5.9         3.2          4.8         1.8 versicolor
## 72           6.1         2.8          4.0         1.3 versicolor
## 73           6.3         2.5          4.9         1.5 versicolor
## 74           6.1         2.8          4.7         1.2 versicolor
## 75           6.4         2.9          4.3         1.3 versicolor
## 76           6.6         3.0          4.4         1.4 versicolor
## 77           6.8         2.8          4.8         1.4 versicolor
## 78           6.7         3.0          5.0         1.7 versicolor
## 79           6.0         2.9          4.5         1.5 versicolor
## 80           5.7         2.6          3.5         1.0 versicolor
## 81           5.5         2.4          3.8         1.1 versicolor
## 82           5.5         2.4          3.7         1.0 versicolor
## 83           5.8         2.7          3.9         1.2 versicolor
## 84           6.0         2.7          5.1         1.6 versicolor
## 85           5.4         3.0          4.5         1.5 versicolor
## 86           6.0         3.4          4.5         1.6 versicolor
## 87           6.7         3.1          4.7         1.5 versicolor
## 88           6.3         2.3          4.4         1.3 versicolor
## 89           5.6         3.0          4.1         1.3 versicolor
## 90           5.5         2.5          4.0         1.3 versicolor
## 91           5.5         2.6          4.4         1.2 versicolor
## 92           6.1         3.0          4.6         1.4 versicolor
## 93           5.8         2.6          4.0         1.2 versicolor
## 94           5.0         2.3          3.3         1.0 versicolor
## 95           5.6         2.7          4.2         1.3 versicolor
## 96           5.7         3.0          4.2         1.2 versicolor
## 97           5.7         2.9          4.2         1.3 versicolor
## 98           6.2         2.9          4.3         1.3 versicolor
## 99           5.1         2.5          3.0         1.1 versicolor
## 100          5.7         2.8          4.1         1.3 versicolor
## 101          6.3         3.3          6.0         2.5  virginica
## 102          5.8         2.7          5.1         1.9  virginica
## 103          7.1         3.0          5.9         2.1  virginica
## 104          6.3         2.9          5.6         1.8  virginica
## 105          6.5         3.0          5.8         2.2  virginica
## 106          7.6         3.0          6.6         2.1  virginica
## 107          4.9         2.5          4.5         1.7  virginica
## 108          7.3         2.9          6.3         1.8  virginica
## 109          6.7         2.5          5.8         1.8  virginica
## 110          7.2         3.6          6.1         2.5  virginica
## 111          6.5         3.2          5.1         2.0  virginica
## 112          6.4         2.7          5.3         1.9  virginica
## 113          6.8         3.0          5.5         2.1  virginica
## 114          5.7         2.5          5.0         2.0  virginica
## 115          5.8         2.8          5.1         2.4  virginica
## 116          6.4         3.2          5.3         2.3  virginica
## 117          6.5         3.0          5.5         1.8  virginica
## 118          7.7         3.8          6.7         2.2  virginica
## 119          7.7         2.6          6.9         2.3  virginica
## 120          6.0         2.2          5.0         1.5  virginica
## 121          6.9         3.2          5.7         2.3  virginica
## 122          5.6         2.8          4.9         2.0  virginica
## 123          7.7         2.8          6.7         2.0  virginica
## 124          6.3         2.7          4.9         1.8  virginica
## 125          6.7         3.3          5.7         2.1  virginica
## 126          7.2         3.2          6.0         1.8  virginica
## 127          6.2         2.8          4.8         1.8  virginica
## 128          6.1         3.0          4.9         1.8  virginica
## 129          6.4         2.8          5.6         2.1  virginica
## 130          7.2         3.0          5.8         1.6  virginica
## 131          7.4         2.8          6.1         1.9  virginica
## 132          7.9         3.8          6.4         2.0  virginica
## 133          6.4         2.8          5.6         2.2  virginica
## 134          6.3         2.8          5.1         1.5  virginica
## 135          6.1         2.6          5.6         1.4  virginica
## 136          7.7         3.0          6.1         2.3  virginica
## 137          6.3         3.4          5.6         2.4  virginica
## 138          6.4         3.1          5.5         1.8  virginica
## 139          6.0         3.0          4.8         1.8  virginica
## 140          6.9         3.1          5.4         2.1  virginica
## 141          6.7         3.1          5.6         2.4  virginica
## 142          6.9         3.1          5.1         2.3  virginica
## 143          5.8         2.7          5.1         1.9  virginica
## 144          6.8         3.2          5.9         2.3  virginica
## 145          6.7         3.3          5.7         2.5  virginica
## 146          6.7         3.0          5.2         2.3  virginica
## 147          6.3         2.5          5.0         1.9  virginica
## 148          6.5         3.0          5.2         2.0  virginica
## 149          6.2         3.4          5.4         2.3  virginica
## 150          5.9         3.0          5.1         1.8  virginica

# Intake ==> ETL ==> [Training Data_idx]

index <- c(base::sample(1:50,25),
           base::sample(51:100,25), 
           base::sample(101:150,25))

# Pipe: data[iris]===> Spark_Neural_Net_Data-Model_Structure{S_NN_DM}

iris_Train = iris[index,]
iris_Test = iris[-index,]

# Pipe: data[iris]===> {S_Train.Test}===>{h2o_S_Train.Test}

iris.h2o_Train <- as.h2o(iris_Train)

##   |                                                                              |                                                                      |   0%  |                                                                              |======================================================================| 100%

iris.h2o_Test <- as.h2o(iris_Test)

##   |                                                                              |                                                                      |   0%  |                                                                              |======================================================================| 100%

# Pipe:data[h2o_.]==>{h2o_S_NN_DM}(epochs:100 ==> 5_nodes_hidden_L1) 

iris_nn_Train <- h2o.deeplearning(x = 1:4 ,y = 5, 
                            training_frame = iris.h2o_Train, 
                            validation_frame = iris.h2o_Test,
                            activation = "Tanh",
                            hidden = c(5), 
                            l1 = 1e-5,
                            epochs = 100, 
                            variable_importances = TRUE)

##   |                                                                              |                                                                      |   0%  |                                                                              |======================================================================| 100%

# Pipe:data[h2o_.]==>{h2o_S_NN_DM}(epochs:100 ==> 5_nodes_hidden_L1) 

iris_nn_Train.cv <- h2o.deeplearning(x = 1:4 ,
                            y = 5, 
                            training_frame = iris.h2o_Train, 
                            validation_frame = iris.h2o_Test,
                            activation = "Tanh",
                            hidden = c(5), 
                            l1 = 1e-5,
                            nfolds = 5,
                            epochs = 100) 

##   |                                                                              |                                                                      |   0%  |                                                                              |===========================================================           |  84%  |                                                                              |======================================================================| 100%

# [Spark_Out]==>{S_tanh[data]} 

plot(x, tanh(x), 
     col = 'lightgreen',
     main = 'Loss Fn Tanh'
     )

# Performance Code ==>Pipe: F{S_NN_DM}[Data]

h2o.performance(iris_nn_Train, train = TRUE)

## H2OMultinomialMetrics: deeplearning
## ** Reported on training data. **
## ** Metrics reported on full training frame **
## 
## Training Set Metrics: 
## =====================
## 
## Extract training frame with `h2o.getFrame("iris_Train_sid_81c2_1")`
## MSE: (Extract with `h2o.mse`) 0.02503523
## RMSE: (Extract with `h2o.rmse`) 0.1582252
## Logloss: (Extract with `h2o.logloss`) 0.1100941
## Mean Per-Class Error: 0.04
## AUC: (Extract with `h2o.auc`) NaN
## AUCPR: (Extract with `h2o.aucpr`) NaN
## AIC: (Extract with `h2o.aic`) NaN
## Confusion Matrix: Extract with `h2o.confusionMatrix(<model>,train = TRUE)`)
## =========================================================================
## Confusion Matrix: Row labels: Actual class; Column labels: Predicted class
##            setosa versicolor virginica  Error     Rate
## setosa         25          0         0 0.0000 = 0 / 25
## versicolor      0         23         2 0.0800 = 2 / 25
## virginica       0          1        24 0.0400 = 1 / 25
## Totals         25         24        26 0.0400 = 3 / 75
## 
## Hit Ratio Table: Extract with `h2o.hit_ratio_table(<model>,train = TRUE)`
## =======================================================================
## Top-3 Hit Ratios: 
##   k hit_ratio
## 1 1  0.960000
## 2 2  1.000000
## 3 3  1.000000

# Performance Code F{S_NN_DM}(.mse)

h2o.mse(iris_nn_Train, train = TRUE)

## [1] 0.02503523

h2o.mse(iris_nn_Train.cv, xval = TRUE)

## [1] 0.03256607

# perfromance Code F{S_NN_DM}(.varimp)

h2o.varimp(iris_nn_Train)

## Variable Importances: 
##       variable relative_importance scaled_importance percentage
## 1 Petal.Length            1.000000          1.000000   0.401914
## 2  Petal.Width            0.668017          0.668017   0.268485
## 3  Sepal.Width            0.491290          0.491290   0.197456
## 4 Sepal.Length            0.328788          0.328788   0.132144

# Predict Code F{S_NN_DM}(.predict)

predictions_Train <- h2o.predict(iris_nn_Train, iris.h2o_Train)

##   |                                                                              |                                                                      |   0%  |                                                                              |======================================================================| 100%

predictions_Train

##   predict    setosa  versicolor    virginica
## 1  setosa 0.9945152 0.005462464 2.231943e-05
## 2  setosa 0.9887402 0.011227083 3.269255e-05
## 3  setosa 0.9949694 0.005016568 1.407947e-05
## 4  setosa 0.9957555 0.004230833 1.364526e-05
## 5  setosa 0.9950313 0.004951623 1.709626e-05
## 6  setosa 0.9936373 0.006338289 2.445173e-05
## 
## [75 rows x 4 columns]

predictions_Test <- h2o.predict(iris_nn_Train, iris.h2o_Test)

##   |                                                                              |                                                                      |   0%  |                                                                              |======================================================================| 100%

predictions_Test 

##   predict    setosa  versicolor    virginica
## 1  setosa 0.9867850 0.013178777 3.618120e-05
## 2  setosa 0.9931986 0.006781547 1.986322e-05
## 3  setosa 0.9946448 0.005335784 1.946356e-05
## 4  setosa 0.9929678 0.007011573 2.065007e-05
## 5  setosa 0.9935740 0.006407667 1.833944e-05
## 6  setosa 0.9960492 0.003935277 1.551795e-05
## 
## [75 rows x 4 columns]

Y_hat_Train = as.factor(as.matrix(predictions_Train$predict))
Y_hat_Test = as.factor(as.matrix(predictions_Test$predict))


# Pipe: data[h2o_.]===> Spark_GBM_Data-Model_Structure{S_GBM_DM}

iris_hex <- as.h2o(iris)

##   |                                                                              |                                                                      |   0%  |                                                                              |======================================================================| 100%

iris_gbm <- h2o.gbm(x = c(1:4), 
                    y = 5, 
                    training_frame = iris.h2o_Train)

##   |                                                                              |                                                                      |   0%  |                                                                              |======================================================================| 100%

Time for this code chunk to run: 11.07 seconds

Take note of the scaled, variable, & relative importance, as well as the percentages applied to each variable. Let’s compare our models:

#DEEP LEARNING MODEL


# Create the learning curve plot 
#[Spark_Out]==>{.learning_curve_plot[h2o_.]} 

learning_curve <- h2o.learning_curve_plot(iris_nn_Train)


# Partial Dependency Graph 3 target classes 
#[Spark_Out]==>{.partialPlot[h2o_.]} 

suppressWarnings(h2o.partialPlot(object = iris_nn_Train, 
                newdata = iris_hex, 
                cols = "Petal.Width",
                targets = c("setosa", "virginica", "versicolor")))

##   |                                                                              |                                                                      |   0%  |                                                                              |======================================================================| 100%

## [[1]]
## PartialDependence: Partial dependency plot for Petal.Width and classes
##  setosa, virginica, versicolor
##    Petal.Width mean_response stddev_response std_error_mean_response
## 1     0.100000      0.367607        0.446107                0.036424
## 2     0.226316      0.360533        0.449881                0.036733
## 3     0.352632      0.354414        0.453039                0.036990
## 4     0.478947      0.349208        0.455554                0.037196
## 5     0.605263      0.344817        0.457431                0.037349
## 6     0.731579      0.341109        0.458708                0.037453
## 7     0.857895      0.337941        0.459442                0.037513
## 8     0.984211      0.335172        0.459695                0.037534
## 9     1.110526      0.332665        0.459518                0.037519
## 10    1.236842      0.330291        0.458922                0.037471
## 11    1.363158      0.327922        0.457864                0.037384
## 12    1.489474      0.325420        0.456237                0.037252
## 13    1.615789      0.322624        0.453888                0.037060
## 14    1.742105      0.319344        0.450634                0.036794
## 15    1.868421      0.315365        0.446281                0.036439
## 16    1.994737      0.310458        0.440647                0.035979
## 17    2.121053      0.304403        0.433575                0.035401
## 18    2.247368      0.297008        0.424944                0.034697
## 19    2.373684      0.288121        0.414659                0.033857
## 20    2.500000      0.277629        0.402625                0.032874
## 
## [[2]]
## PartialDependence: Partial dependency plot for Petal.Width and classes
##  setosa, virginica, versicolor
##    Petal.Width mean_response stddev_response std_error_mean_response
## 1     0.100000      0.021313        0.052868                0.004317
## 2     0.226316      0.029265        0.071026                0.005799
## 3     0.352632      0.039723        0.091920                0.007505
## 4     0.478947      0.053171        0.115398                0.009422
## 5     0.605263      0.070057        0.141265                0.011534
## 6     0.731579      0.090717        0.169087                0.013806
## 7     0.857895      0.115317        0.198144                0.016178
## 8     0.984211      0.143853        0.227425                0.018569
## 9     1.110526      0.176229        0.255777                0.020884
## 10    1.236842      0.212325        0.282247                0.023045
## 11    1.363158      0.251941        0.306375                0.025015
## 12    1.489474      0.294605        0.328161                0.026794
## 13    1.615789      0.339391        0.347801                0.028398
## 14    1.742105      0.384905        0.365461                0.029840
## 15    1.868421      0.429483        0.381183                0.031123
## 16    1.994737      0.471523        0.394891                0.032243
## 17    2.121053      0.509790        0.406560                0.033195
## 18    2.247368      0.543530        0.416352                0.033995
## 19    2.373684      0.572455        0.424458                0.034657
## 20    2.500000      0.596743        0.430829                0.035177
## 
## [[3]]
## PartialDependence: Partial dependency plot for Petal.Width and classes
##  setosa, virginica, versicolor
##    Petal.Width mean_response stddev_response std_error_mean_response
## 1     0.100000      0.611080        0.434954                0.035514
## 2     0.226316      0.610202        0.435208                0.035535
## 3     0.352632      0.605863        0.434131                0.035447
## 4     0.478947      0.597621        0.431738                0.035251
## 5     0.605263      0.585127        0.428097                0.034954
## 6     0.731579      0.568174        0.423274                0.034560
## 7     0.857895      0.546742        0.417274                0.034070
## 8     0.984211      0.520975        0.409900                0.033468
## 9     1.110526      0.491107        0.400615                0.032710
## 10    1.236842      0.457384        0.388592                0.031728
## 11    1.363158      0.420137        0.373003                0.030456
## 12    1.489474      0.379975        0.353352                0.028851
## 13    1.615789      0.337985        0.329634                0.026915
## 14    1.742105      0.295751        0.302355                0.024687
## 15    1.868421      0.255152        0.272438                0.022245
## 16    1.994737      0.218019        0.241075                0.019684
## 17    2.121053      0.185807        0.209812                0.017131
## 18    2.247368      0.159462        0.181007                0.014779
## 19    2.373684      0.139424        0.158010                0.012901
## 20    2.500000      0.125628        0.144303                0.011782

# Variable Importance Plot 
#[Spark_Out]==>{.permutation_importance_plot[h2o_.]}

suppressWarnings(h2o.permutation_importance_plot(iris_nn_Train, iris.h2o_Train))

# Learning Curve Graph 
# [Spark_Out]==>{(.print)[.learning_curve_plot[h2o_.]

suppressWarnings(print(learning_curve))

Time for this code chunk to run: 1.27 seconds

Learning curves show error metric dependence on learning progress, e.g., RMSE vs number of trees trained so far in GBM. There can be up to 4 curves showing Training, Validation, Training on CV Models, and Cross-validation error.

Partial dependence plot gives a graphical depiction of the marginal effect of a variable on the response. The effect of a variable is measured in change in the mean response. Note: Unlike random Forest’s partial Plot when plotting partial dependence the mean response (probabilities) is returned rather than the mean of the log class probability.

#GBM MODEL

# Create the learning curve plot 
#[Spark_Out]==>{.learning_curve_plot[h2o_.]} 

learning_curve <- h2o.learning_curve_plot(iris_gbm)

suppressWarnings(print(learning_curve))

# Partial Dependency Graph 3 target classes 
#[Spark_Out]==>{.partialPlot[h2o_.]} 

suppressWarnings(h2o.partialPlot(object = iris_gbm, 
                newdata = iris_hex, 
                cols = "Petal.Length", 
                targets = c("setosa")))

##   |                                                                              |                                                                      |   0%  |                                                                              |======================================================================| 100%

## PartialDependence: Partial dependency plot for Petal.Length and class setosa
##    Petal.Length mean_response stddev_response std_error_mean_response
## 1      1.000000      0.874189        0.173956                0.014203
## 2      1.310526      0.874189        0.173956                0.014203
## 3      1.621053      0.874189        0.173956                0.014203
## 4      1.931579      0.874189        0.173956                0.014203
## 5      2.242105      0.874189        0.173956                0.014203
## 6      2.552632      0.874189        0.173956                0.014203
## 7      2.863158      0.178106        0.200938                0.016407
## 8      3.173684      0.090476        0.097154                0.007933
## 9      3.484211      0.090476        0.097154                0.007933
## 10     3.794737      0.090476        0.097154                0.007933
## 11     4.105263      0.069856        0.069049                0.005638
## 12     4.415789      0.069288        0.068693                0.005609
## 13     4.726316      0.065703        0.062703                0.005120
## 14     5.036842      0.054115        0.073618                0.006011
## 15     5.347368      0.054358        0.073928                0.006036
## 16     5.657895      0.047911        0.065060                0.005312
## 17     5.968421      0.040208        0.054391                0.004441
## 18     6.278947      0.040208        0.054391                0.004441
## 19     6.589474      0.040208        0.054391                0.004441
## 20     6.900000      0.040208        0.054391                0.004441

suppressWarnings(h2o.partialPlot(object = iris_gbm, 
                newdata = iris_hex, 
                cols = "Petal.Width",
                targets = c("setosa", "virginica", "versicolor")))

##   |                                                                              |                                                                      |   0%  |                                                                              |======================================================================| 100%

## [[1]]
## PartialDependence: Partial dependency plot for Petal.Width and classes
##  setosa, virginica, versicolor
##    Petal.Width mean_response stddev_response std_error_mean_response
## 1     0.100000      0.379974        0.441103                0.036016
## 2     0.226316      0.379974        0.441103                0.036016
## 3     0.352632      0.378724        0.441707                0.036065
## 4     0.478947      0.378724        0.441707                0.036065
## 5     0.605263      0.378724        0.441707                0.036065
## 6     0.731579      0.378724        0.441707                0.036065
## 7     0.857895      0.308652        0.436679                0.035655
## 8     0.984211      0.298318        0.422488                0.034496
## 9     1.110526      0.298318        0.422488                0.034496
## 10    1.236842      0.298625        0.422901                0.034530
## 11    1.363158      0.298625        0.422901                0.034530
## 12    1.489474      0.298625        0.422901                0.034530
## 13    1.615789      0.298625        0.422901                0.034530
## 14    1.742105      0.337349        0.436638                0.035651
## 15    1.868421      0.337317        0.436662                0.035653
## 16    1.994737      0.337320        0.436660                0.035653
## 17    2.121053      0.337149        0.436786                0.035663
## 18    2.247368      0.337149        0.436786                0.035663
## 19    2.373684      0.337149        0.436786                0.035663
## 20    2.500000      0.337149        0.436786                0.035663
## 
## [[2]]
## PartialDependence: Partial dependency plot for Petal.Width and classes
##  setosa, virginica, versicolor
##    Petal.Width mean_response stddev_response std_error_mean_response
## 1     0.100000      0.273976        0.408173                0.033327
## 2     0.226316      0.273976        0.408173                0.033327
## 3     0.352632      0.273247        0.407347                0.033260
## 4     0.478947      0.273247        0.407347                0.033260
## 5     0.605263      0.273247        0.407347                0.033260
## 6     0.731579      0.273247        0.407347                0.033260
## 7     0.857895      0.267080        0.407082                0.033238
## 8     0.984211      0.261006        0.401017                0.032743
## 9     1.110526      0.261006        0.401017                0.032743
## 10    1.236842      0.259442        0.400870                0.032731
## 11    1.363158      0.259442        0.400870                0.032731
## 12    1.489474      0.259442        0.400870                0.032731
## 13    1.615789      0.259442        0.400869                0.032731
## 14    1.742105      0.420001        0.420939                0.034370
## 15    1.868421      0.420199        0.421206                0.034391
## 16    1.994737      0.420198        0.421206                0.034391
## 17    2.121053      0.419831        0.422100                0.034464
## 18    2.247368      0.419831        0.422100                0.034464
## 19    2.373684      0.419831        0.422100                0.034464
## 20    2.500000      0.419831        0.422100                0.034464
## 
## [[3]]
## PartialDependence: Partial dependency plot for Petal.Width and classes
##  setosa, virginica, versicolor
##    Petal.Width mean_response stddev_response std_error_mean_response
## 1     0.100000      0.346050        0.415938                0.033961
## 2     0.226316      0.346050        0.415938                0.033961
## 3     0.352632      0.348028        0.417041                0.034051
## 4     0.478947      0.348028        0.417041                0.034051
## 5     0.605263      0.348028        0.417041                0.034051
## 6     0.731579      0.348028        0.417041                0.034051
## 7     0.857895      0.424268        0.444052                0.036257
## 8     0.984211      0.440676        0.434542                0.035480
## 9     1.110526      0.440676        0.434542                0.035480
## 10    1.236842      0.441933        0.435035                0.035520
## 11    1.363158      0.441933        0.435035                0.035520
## 12    1.489474      0.441933        0.435035                0.035520
## 13    1.615789      0.441933        0.435035                0.035520
## 14    1.742105      0.242650        0.332170                0.027122
## 15    1.868421      0.242484        0.332286                0.027131
## 16    1.994737      0.242482        0.332284                0.027131
## 17    2.121053      0.243019        0.333555                0.027235
## 18    2.247368      0.243019        0.333555                0.027235
## 19    2.373684      0.243019        0.333555                0.027235
## 20    2.500000      0.243019        0.333555                0.027235

# Variable Importance Plot 
#[Spark_Out]==>{.permutation_importance_plot[h2o_.]}

suppressWarnings(h2o.permutation_importance_plot(iris_gbm, iris.h2o_Train))

Time for this code chunk to run: 1.29 seconds

VI. Optimizing Hyper-Parameters

A model hyperparameter is a characteristic of a model that is external to the model and whose value cannot be estimated from data. The value of the hyperparameter has to be set before the learning process begins. For example, c in Support Vector Machines, k in k-Nearest Neighbors, the number of hidden layers in Neural Networks.

• Random Search. Define a search space as a bounded domain of hyperparameter values and randomly sample points in that domain.
• Grid Search. Define a search space as a grid of hyperparameter values and evaluate every position in the grid.

In the following example we will manually build out an example grid_search to optimize our hyperparameters.

#Deep Learning Grid Search 

hc <- H2OContext.getOrCreate(h2oConf)

# Pipe:Create:data[h2o_.]==>{h2o_S_List}(Grid:.h2o.hidden ==>  hyper_params:.h2o) 

hidden_opt <- list(c(1), c(2), c(3), 4,5,6,7,8,9,10, c(3,4),c(4,4), c(5,4), c(6,4))

hyper_params <- list(hidden = hidden_opt)

# data in H2O format
# Pipe:data[h2o_.]==>{h2o_S_Deep_LNN_DM}(Grid:.h2o ==>  hyper_params:.h2o) 

model_grid <- h2o.grid(
                        "deeplearning",
                        
                        hyper_params = hyper_params,
                        x = 1:4 ,
                        y = 5, 
                        training_frame = iris.h2o_Train, 
                        validation_frame = iris.h2o_Test,
                        activation = "Tanh",
                        seed = 19, 
                        reproducible = TRUE, 
                        nfolds = 5
                        )

##   |                                                                              |                                                                      |   0%Error in .h2o.doSafeREST(h2oRestApiVersion = h2oRestApiVersion, urlSuffix = urlSuffix,  : 
##   Unexpected CURL error: Failed to connect to 127.0.0.1 port 54324 after 0 ms: Couldn't connect to server
## [1] "Job request failed Unexpected CURL error: Failed to connect to 127.0.0.1 port 54324 after 0 ms: Couldn't connect to server, will retry after 3s."
##   |                                                                              |======================================================================| 100%

model_grid

## H2O Grid Details
## ================
## 
## Grid ID: Grid_DeepLearning_iris_Train_sid_81c2_1_model_R_1720774150565_4 
## Used hyper parameters: 
##   -  hidden 
## Number of models: 14 
## Number of failed models: 0 
## 
## Hyper-Parameter Search Summary: ordered by increasing logloss
##    hidden
## 1      10
## 2       8
## 3       7
## 4  [5, 4]
## 5  [4, 4]
## 6  [6, 4]
## 7       6
## 8       9
## 9  [3, 4]
## 10      5
## 11      3
## 12      4
## 13      2
## 14      1
##                                                                   model_ids
## 1  Grid_DeepLearning_iris_Train_sid_81c2_1_model_R_1720774150565_4_model_10
## 2   Grid_DeepLearning_iris_Train_sid_81c2_1_model_R_1720774150565_4_model_8
## 3   Grid_DeepLearning_iris_Train_sid_81c2_1_model_R_1720774150565_4_model_7
## 4  Grid_DeepLearning_iris_Train_sid_81c2_1_model_R_1720774150565_4_model_13
## 5  Grid_DeepLearning_iris_Train_sid_81c2_1_model_R_1720774150565_4_model_12
## 6  Grid_DeepLearning_iris_Train_sid_81c2_1_model_R_1720774150565_4_model_14
## 7   Grid_DeepLearning_iris_Train_sid_81c2_1_model_R_1720774150565_4_model_6
## 8   Grid_DeepLearning_iris_Train_sid_81c2_1_model_R_1720774150565_4_model_9
## 9  Grid_DeepLearning_iris_Train_sid_81c2_1_model_R_1720774150565_4_model_11
## 10  Grid_DeepLearning_iris_Train_sid_81c2_1_model_R_1720774150565_4_model_5
## 11  Grid_DeepLearning_iris_Train_sid_81c2_1_model_R_1720774150565_4_model_3
## 12  Grid_DeepLearning_iris_Train_sid_81c2_1_model_R_1720774150565_4_model_4
## 13  Grid_DeepLearning_iris_Train_sid_81c2_1_model_R_1720774150565_4_model_2
## 14  Grid_DeepLearning_iris_Train_sid_81c2_1_model_R_1720774150565_4_model_1
##    logloss
## 1  0.43372
## 2  0.48605
## 3  0.49865
## 4  0.52968
## 5  0.57270
## 6  0.58271
## 7  0.59055
## 8  0.61162
## 9  0.70005
## 10 0.78003
## 11 0.93817
## 12 1.00948
## 13 1.47813
## 14 3.25693

Best_Grid_Model <- h2o.getModel(model_grid@model_ids[[1]])

# Performance Code F{S_NN_DM}(.mse)

h2o.mse(Best_Grid_Model, xval = TRUE)

## [1] 0.1424292

h2o.performance(model = Best_Grid_Model , newdata = iris.h2o_Test)

## H2OMultinomialMetrics: deeplearning
## 
## Test Set Metrics: 
## =====================
## 
## MSE: (Extract with `h2o.mse`) 0.09396421
## RMSE: (Extract with `h2o.rmse`) 0.3065358
## Logloss: (Extract with `h2o.logloss`) 0.3041457
## Mean Per-Class Error: 0.1466667
## AUC: (Extract with `h2o.auc`) NaN
## AUCPR: (Extract with `h2o.aucpr`) NaN
## AIC: (Extract with `h2o.aic`) NaN
## Confusion Matrix: Extract with `h2o.confusionMatrix(<model>, <data>)`)
## =========================================================================
## Confusion Matrix: Row labels: Actual class; Column labels: Predicted class
##            setosa versicolor virginica  Error      Rate
## setosa         25          0         0 0.0000 =  0 / 25
## versicolor      0         19         6 0.2400 =  6 / 25
## virginica       0          5        20 0.2000 =  5 / 25
## Totals         25         24        26 0.1467 = 11 / 75
## 
## Hit Ratio Table: Extract with `h2o.hit_ratio_table(<model>, <data>)`
## =======================================================================
## Top-3 Hit Ratios: 
##   k hit_ratio
## 1 1  0.853333
## 2 2  1.000000
## 3 3  1.000000

# Variable Importance Plot 
#[Spark_Out]==>{.permutation_importance_plot[h2o_.]}

varimp <- h2o.varimp(Best_Grid_Model)
varimp

## Variable Importances: 
##       variable relative_importance scaled_importance percentage
## 1 Petal.Length            1.000000          1.000000   0.311023
## 2  Sepal.Width            0.922365          0.922365   0.286877
## 3  Petal.Width            0.800324          0.800324   0.248920
## 4 Sepal.Length            0.492504          0.492504   0.153180

Time for this code chunk to run: 162.64 seconds

Above: We created 14 different model configurations with various network architectures, with model_14 outperforming all other models in terms of optimizing log loss, while also analyzing variable importance. Lets look & compare this version with our previous versions using the same visuals.

hc <- H2OContext.getOrCreate(h2oConf)

# Variable Importance Plot 
#[Spark_Out]==>{.permutation_importance_plot[.Grid.h2o_.]}

varimp <- h2o.varimp(Best_Grid_Model)
varimp

## Variable Importances: 
##       variable relative_importance scaled_importance percentage
## 1 Petal.Length            1.000000          1.000000   0.311023
## 2  Sepal.Width            0.922365          0.922365   0.286877
## 3  Petal.Width            0.800324          0.800324   0.248920
## 4 Sepal.Length            0.492504          0.492504   0.153180

# Create the learning curve plot 
#[Spark_Out]==>{.learning_curve_plot[h2o_.]} 

learning_curve <- h2o.learning_curve_plot(Best_Grid_Model)
suppressWarnings(print(learning_curve))

# Variable Importance Plot 
#[Spark_Out]==>{.permutation_importance_plot[h2o_.]}

suppressWarnings(h2o.varimp_plot(Best_Grid_Model))

# Partial Dependency Graph 3 target classes 
#[Spark_Out]==>{.partialPlot[h2o_.]} 

suppressWarnings(h2o.partialPlot(object = Best_Grid_Model, 
                newdata = iris_hex, 
                cols = "Petal.Width",
                targets = c("setosa", "virginica", "versicolor")))

##   |                                                                              |                                                                      |   0%  |                                                                              |======================================================================| 100%

## [[1]]
## PartialDependence: Partial dependency plot for Petal.Width and classes
##  setosa, virginica, versicolor
##    Petal.Width mean_response stddev_response std_error_mean_response
## 1     0.100000      0.725999        0.272394                0.022241
## 2     0.226316      0.675680        0.293207                0.023940
## 3     0.352632      0.623271        0.311085                0.025400
## 4     0.478947      0.570259        0.324833                0.026523
## 5     0.605263      0.517296        0.332802                0.027173
## 6     0.731579      0.463131        0.332564                0.027154
## 7     0.857895      0.402971        0.319441                0.026082
## 8     0.984211      0.328495        0.286459                0.023389
## 9     1.110526      0.239321        0.234876                0.019178
## 10    1.236842      0.156017        0.182503                0.014901
## 11    1.363158      0.095800        0.137812                0.011252
## 12    1.489474      0.055929        0.097988                0.008001
## 13    1.615789      0.030266        0.064726                0.005285
## 14    1.742105      0.015181        0.039260                0.003206
## 15    1.868421      0.007293        0.021499                0.001755
## 16    1.994737      0.003535        0.011216                0.000916
## 17    2.121053      0.001797        0.006088                0.000497
## 18    2.247368      0.000952        0.003394                0.000277
## 19    2.373684      0.000519        0.001859                0.000152
## 20    2.500000      0.000295        0.001041                0.000085
## 
## [[2]]
## PartialDependence: Partial dependency plot for Petal.Width and classes
##  setosa, virginica, versicolor
##    Petal.Width mean_response stddev_response std_error_mean_response
## 1     0.100000      0.000386        0.001124                0.000092
## 2     0.226316      0.000611        0.001518                0.000124
## 3     0.352632      0.001112        0.002771                0.000226
## 4     0.478947      0.002334        0.006144                0.000502
## 5     0.605263      0.005474        0.013676                0.001117
## 6     0.731579      0.013827        0.028903                0.002360
## 7     0.857895      0.035914        0.061275                0.005003
## 8     0.984211      0.087525        0.129468                0.010571
## 9     1.110526      0.177099        0.230964                0.018858
## 10    1.236842      0.286331        0.316897                0.025875
## 11    1.363158      0.402395        0.349497                0.028536
## 12    1.489474      0.533425        0.333501                0.027230
## 13    1.615789      0.672618        0.284730                0.023248
## 14    1.742105      0.794699        0.218665                0.017854
## 15    1.868421      0.881800        0.156098                0.012745
## 16    1.994737      0.934135        0.111287                0.009087
## 17    2.121053      0.962880        0.078757                0.006430
## 18    2.247368      0.978728        0.051255                0.004185
## 19    2.373684      0.987645        0.029718                0.002426
## 20    2.500000      0.992575        0.016298                0.001331
## 
## [[3]]
## PartialDependence: Partial dependency plot for Petal.Width and classes
##  setosa, virginica, versicolor
##    Petal.Width mean_response stddev_response std_error_mean_response
## 1     0.100000      0.273614        0.272453                0.022246
## 2     0.226316      0.323709        0.293359                0.023953
## 3     0.352632      0.375617        0.311486                0.025433
## 4     0.478947      0.427408        0.325928                0.026612
## 5     0.605263      0.477230        0.335859                0.027423
## 6     0.731579      0.523042        0.341205                0.027859
## 7     0.857895      0.561115        0.343597                0.028055
## 8     0.984211      0.583980        0.347875                0.028404
## 9     1.110526      0.583580        0.358951                0.029308
## 10    1.236842      0.557652        0.368064                0.030052
## 11    1.363158      0.501805        0.359335                0.029340
## 12    1.489474      0.410646        0.327874                0.026771
## 13    1.615789      0.297116        0.275835                0.022522
## 14    1.742105      0.190120        0.210700                0.017204
## 15    1.868421      0.110907        0.149684                0.012222
## 16    1.994737      0.062330        0.106146                0.008667
## 17    2.121053      0.035323        0.074958                0.006120
## 18    2.247368      0.020320        0.048823                0.003986
## 19    2.373684      0.011836        0.028338                0.002314
## 20    2.500000      0.007129        0.015525                0.001268

Time for this code chunk to run: 1.06 seconds

We have plenty of data to ensure that we both understand & implement an optimized model. In addition h2o can be used to create dashboards & apps. Auto Doc allows you to create automatic model documentation for your models created in either H2O or Driver less AI

H2O Driver less AI: Like H2O, this tool also offers automatic machine learning but this tool takes it a few steps further. As well as trying different machine learning algorithms (and ensembles of the available algorithms), this tool will also perform automatic feature engineering, produce data visualizations and post training diagnostics plots, and give performance metrics for each model

H2O Auto Doc: Auto Doc allows you to create automatic model documentation for your models created in either H2O or Driver less AI (this feature is integrated into Driver less AI). You can also use this tool on any model you create using the Python library ScikitLearn.

H2O MLOps: If you are looking at putting your machine learning models into production this is where MLOps (machine learning operations) can help. H2O MLOps can be used to deploy models that you have created in both H2O and H2O Driver less AI and allows you to easily maintain them once they are in production. The tool uses Kubernetes for easy deployment, scaling and management 

We had one error out of 75 classifying a versicolor as a virginica. Compare that with the results from the very first NN model displayed at the very begging where we had 3 errors classifying a versicolor as a virginica & classifying a 2 virginica’s as a versicolor’s.

It can get tedious running all the different kinds of models. However H2O supports two types of grid search – traditional (or “cartesian”) grid search and random grid search. In a Cartesian grid search, users specify a set of values for each hyper-parameter that they want to search over, and H2O will train a model for every combination of the hyper-parameter values. In random grid search, the user specifies the hyper-parameter space in the exact same way, except H2O will sample uniformly from the set of all possible hyper-parameter value combinations. In random grid search, the user also specifies a stopping criterion, which controls when the random grid search is completed.

Continued on DATA SCIENCE, ARCHITECTURE, AI and ML Part II